Design and Performance Evaluation of a Cache Assist to implement Selective Caching

EEcient instruction and data caches are extremely important for achieving good performance from modern high performance processors. Conventional cache architectures exploit locality, but do so rather blindly. By forcing all references through a single structure, the cache's eeectiveness on many references is reduced. This paper presents a selective caching scheme for improving cache performance, implemented using a cache assist namely the annex cache. Except for lling a main cache at cold start, all entries come to the cache via the annex cache. A block from the annex cache gets swapped with a main cache block only if it has been referenced twice after the connicting main cache block was referenced. Essentially, low usage items are not allowed to create connict misses in the main cache. Items referenced only rarely will be excluded from the main cache, eliminating several connict misses and swaps. The basic premise is that an item deserves to be in the main cache only if it can prove its right to exist in the main cache by demonstrating locality. The annex cache has some of the features of a victim cache, but the processor can access annex cache entries directly, i. e. annex cache entries can bypass the main cache. Extensive simulation studies for annex and victim caches using a variety of SPEC programs are presented in the paper. Annex caches were observed to be signiicantly better than conventional caches and comparable to victim caches, while providing hardware advantages in comparison to victim caches.